Chen Yulin's BlogChen Yulin's Blog
HomeArchivesCategoriesTagsAbout
  目录
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Posted 2024-09-27Updated 2025-05-08Reviewa minute read (About 112 words)   visits

Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling

Background: RNN

首先介绍了RNN通过hidden state来实现记忆力功能

但指出RNN的训练有梯度消失/爆炸的现象,且记忆会沿序列长度的增加而指数下降,缺乏长期记忆能力。 解决梯度消失/爆炸目前有梯度裁剪和二阶梯度的方法,但成效并不显著

Gated RNN

[[On the Properties of Neural Machine Translation= Encoder–Decoder Approaches]]

Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling

http://chen-yulin.github.io/2024/09/27/[OBS]Deep Learning-RNN-Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling/

Author

Chen Yulin

Posted on

2024-09-27

Updated on

2025-05-08

Licensed under

#Research-paperNLPRNNGated-NN
On the Properties of Neural Machine Translation= Encoder–Decoder Approaches
Attention Is All You Need

Comments

Chen Yulin

Chen Yulin

SJTU student

Manchester by the Sea

Posts

259

Categories

8

Tags

187

Follow

Catalogue

  • Background: RNN
  • Gated RNN

Archives

  • May 20255
  • April 202517
  • March 202545
  • February 202512
  • January 202513
  • December 202412
  • November 20244
  • October 202418
  • September 202417
  • August 202413
  • July 20243
  • June 20245
  • May 202413
  • April 202417
  • March 20241
  • January 20241
  • December 20231
  • May 202346
  • August 20221
  • May 20226
  • April 20229

Recents

Feature Pyramid Networks for Object Detection

2025-05-08

Feature Pyramid Networks for Object Detection

Review

Write Latex in Neovim on Archlinux

2025-05-07

Write Latex in Neovim on Archlinux

Note

Davinci-resolve on Archlinux

2025-05-07

Davinci-resolve on Archlinux

Note

Deformable Convolutional Networks

2025-05-06

Deformable Convolutional Networks

Review

2025-05-01

2025 Summer Schedule

Schedule

Tags

3D-Scene4
6-D3
AI10
AIGC1
AR2
Academic1
Algorithm1
Aliyun1
App2
Atlas1
BS41
Beautify1
Behaviorism1
Business1
C1
CADC1
CD1
CLIP5
CNN1
CV27
Capstone10
Communication2
Contrastive-Learning3
Control2
Csharp9
Css1
Cuda3
DD1
DINO4
DT1
Dataframe1
Debate5
Debugger1
Diffusion1
Discrete-Mathematics1
Docker1
Docs2
Dynamic-programming1
ESP322
Education1
Embeded-System9
Embodied-AI8
Emoation1
Emotion12
Ethic1
FL1
FPN2
Family1
Federated-Learning1
Foundation1
Functional programming1
GPT3
Game5
Gated-NN2
Git7
Github1
Godot3
HPC1
HRI2
Haskell1
Health2
Hexo10
Hierarchical1
Html5
Humanism1
Hyprland2
IK1
Image-Grounding1
Image-Text5
Image-generation1
ImitationLearning3
Jolt1
Json1
LLM12
LSP2
Latex2
Life4
LinearAlgebra1
Linux20
Live2d1
Love3
Lua1
MBTI1
ML5
MR/AR3
Mason1
Math3
Meme1
Message-Passing1
Mod3
Motivation1
Movie1
Multi-modal6
Multi-view1
Music5
NLP4
NN4
Network2
Nodejs5
Numpy1
Nvim9
Object-Detection4
Open-Vocabulary9
OpenCV1
Oral1
PHD1
PSY5
Pandas2
Panoptic1
Path1
Philosophy3
PhysX1
Physical-Scene4
Physics-engine1
Pio2
Planning1
Plugin8
PoseEstimation3
Postgraduate1
Prefab1
Probability1
Python26
Pytorch1
QML1
Quantum1
RNN4
ROS3
Reading19
Real2Sim1
Reconstruct9
Regex2
Reinforcement-learning1
Repository5
Representation-Learning1
Research-paper86
Robot1
Robotics16
SJTU-Lecture1
SQL2
SSH2
Scene-graph29
Scene-synthesis1
Science-fiction1
Scrap1
Script2
Segmentation7
Semantic12
Shader3
Shell4
Signals and Systems1
Sim2Real1
Sklearn1
Snippets1
Society4
Star-rail1
Subgraph1
Submodule1
Supervised-learning2
Survey3
TC1
TOEFL1
Task-Planning6
Tasks4
Tech Communication1
Torch4
Transformer11
Translation-Embedding2
Travel2
Unity20
Unsupervised-learning1
VLM5
VLP2
Version-management1
ViT4
VideoEditing2
Vim1
Visual-Relation20
WSL1
Waybar1
Wayland1
Web1
Website1
Well-being1
Window-manager2
YKLL3
Zen2
🐱1
Chen Yulin's BlogChen Yulin's Blog

© 2025 Chen Yulin  Powered by Hexo & Icarus

×